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Abstract 






Qualifications Based Selection Policy Implications for 

Naval Facilities 

by 

Gregory Alfred Garcia, M.S.E. 

The University of Texas at Austin, 1996 

Supervisor: G. Edward Gibson, Jr. 

For years the construction industry has struggled with hiring qualified 
Architect and Engineering (A/E) firms. The question proposed today is if the 
benefits derived from going through a long, laborious process worth the added 
time and money spent? The United States deemed this concept so crucial that the 
Brooks Act was implemented in 1972. Essentially, the Act mandates that all 
design work must be awarded based on qualifications of the A/E firms rather than 
price. In today’s construction industry, the private sector has begin to apply some 
price pressures on A/E firms. This study will focus on comparing the overall 
success of office building projects that have used purely the qualifications versus 
projects that have used some price considerations in the selection process of their 
A/E firms. 
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Chapter 1: Introduction 




1.1 Purpose 

For over 25 years the Government and many in of the private construction 
industry have been using the Brooks Act, a qualification-based selection (QBS) 
process, to hire A/E services for construction projects. The consensus from many 
design professionals is that the quality of a project is directly related to the method 
in which the firm is selected. Although this feeling is shared by many A/E 
designers, there has never been any significant data gathered to confirm the 
benefits of the Brooks Act. The Act was adopted in the 1970’s not only by 
federal agencies but also by all 50 states. Since then three states, Florida, 
Maryland and Massachusetts, have changed their laws allowing owners to choose 
design services based on price alone. Maryland has since reverted back to QBS. 
(Vanden Bosch et al. 1996) 

According to the laws in most states, the sole method of contracting with 
an A/E firms for public projects should be QBS. However, reports from within the 
construction industry have shown a significant trend that many owners are using 
price to determine selection of A/E services. This brings up two key questions; 
What degree of difficulty and how successful are these projects?, and how much 
did price control the selection of the design services? The Construction Industry 
Institute has done a preliminary study that set up the framework for an intensive 
study to be done in this area. (Vanden Bosch et al. 1996) 

Among the many issues this study addressed, are the various types of A/E 
selection methods. From this study, Table 1.1 gives 5 methods of A/E selection. 
The first three methods are categorized as “price-based” selection and the last two 

1 



are QBS. As project data are analyzed using the Questionnaires generated by 
phase I of the CII investigation, the projects will be separated into the two 
categories. This will allow a thorough analysis to be conducted on QBS versus 
non-QBS projects. The final goal using the data obtained from randomly selected 
projects is to determine whether the QBS process provides any significant impact 
to the overall success of the projects. 


Table 1.1 A/E Selection Methods (Vanden Bosch et al. 1996) 



DESCRIPTION 

Non-OBS 

Bid 

{Price) 

Price is the only selection determinate. 

Combination 

(Pre-qualification, Price) 

Two 

envelope 

The first envelope contains qualifications only. All highly 
qualified firms are pre-qualified. Pre-qualified firms are invited 
to submit price proposals. The second envelope contains the 
price proposal. Award is based on best price. 

One 

envelope 

(Best value, considering qualifications and price) 

Firms simultaneously submit qualifications and price. Award 
may be based on some combination of price and qualifications. 

QBS 

Two 

envelope 

The first envelope contains qualifications only. The one best 
qualified firm is selected and asked to submit a price proposal. 
Price is negotiated with the one best qualified firm only. 

Sole source 

(Reputation is the selection determinant) 

The firm is selected based on their reputation and/or familiarity 
with client’s project. Price may or may not be negotiated. 


1.2 Objectives 

Naval Facilities Command (NAVFAC) spends millions of dollars a year 
contracting private A/E sources. The Phase I report has discovered a growing 














trend in the private sector to consider price more of a factor during the A/E 
selection process. The primary focus of this study is to determine if there is any 
benefit to these new selection processes. Is the construction industry saving 
money and getting the same quality facility? One of NAVFAC’s goals is to 
continually search for better ways to provide support to the Fleet. This study will 
provide unbiased information and possibly a way for NAVFAC to provide A/E 
services faster and cheaper to their customers. The four objectives for this study 
are: 

1 ) Firmly establish NAVFAC’s position with respect to the Brooks Act. 

2 ) Analyze randomly selected project data to determine if QBS versus 

non-QBS of design service have an impact on final outcome of a 

construction project. 

3) Critique the questionnaire developed by CII. 

4) Outline possible policy implications for NAVFAC based on this study. 

1.3 Structure of Thesis 

The following chapters will present background information on the Navy’s 
current policy with respect to the Brooks Act and CII’s Phase I report which 
includes a thorough literature review. Once the background has been established, 
the remaining chapters will include the research methodology, presentation and 
data analysis of a pilot study, as well as conclusions and policy implications for 
NAVFAC. 
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Chapter 2 Background 


2.1 Introduction to Navy A/E Contracting 

Over the past decade, there has been a move to improve methods in 
contracting for facility design. The emphasis has been placed on providing better 
services without compromising quality of design and construction. This concept 
coupled with the draw down of the military has forced the Government to look at 
new and innovative methods in the contracting arena. One of the areas under 
scrutiny is outsourcing of design to A/E firms. 

There appears to be a transition phase in effect within NAVFAC with an 
increased use of Design/Build projects. One study revealed Design/Build projects 
on child care facilities realized a $20 per square foot (SF), 75 percent reduction in 
the number of design changes, and completed the projects 8 months earlier than 
the typical Design/Bid/Build projects. (Roth 1995) These are significant results, 
although the study was performed on a small sample of projects. Not only does 
the owner get the use of the facility earlier at less cost, but also the contracting 
administration team can focus on other projects. This reinforces the concept of 
working more efficiently with less work force. 

If Design/Build contracting seems so appealing, then why is the transition 
in using this process occurring at a relatively slow pace? Naval Facilities 
Engineering Command (NAVFACENGCOM) consists of thousands of people 
such as engineers, lawyers, contracting officers, contract administrators and 
inspectors who all play a major role and work in different offices around the world. 
Not only must thousands of people be trained to administer Design/Build 
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contracts, but also the legal aspects need to be reviewed so that free and open 
competition is not compromised. In addition, the advantages and disadvantages 
must be analyzed to help in determining if Design/Build is a viable long term 
solution. Tables 2.1 and 2.2 summarize some of the advantages and disadvantages 
associated with Design/Build contracting. (Roth 1995) 


Table 2.1 Design/Build Advantages (Roth 1995) 


Area of 
Impact 

Design/Build Advantages 

A Time 

• Use of fast-track concepts allows project to be completed more 
quickly. 

• Project can be prepared for solicitation and awarded quickly. 

• Design/Build has been proven to be 30% faster at delivering the 
project in some studies. 

Cost 

• Guaranteed maximum price is established early in the process. 

• Number of modifications significantly reduced. 

• In-house staff can be effectively used for IFB development. j 

• Method recognizes the increased importance of the time-value 
of financing and incorporates fast-track well. 

• Method enhances the effectiveness and incorporation of TQM, 
partnering, team-building and fast-tracking concepts. 

Coordination 

• Single entity responsible for design and construction. 

• Close coordination inherently required by all parties leads to 
quick problem resolution. 

• Close coordination between A/E and Contractor occurs 
regarding design feasibility and constructability issues. 

• Design/Build involves Subcontractors earlier in the process 
obtaining valuable design input. 

• A/E designs to contractor’s strengths facilitating construction . 














Table 2.1 Design/Build Advantages (Cont.) (Roth 1995] 


Area of Impact 

Design/Build Advantages 

Coordination (Cont.) 

• The new organizational make up within Design/Build 

organization maximizes the respective talents and experience 
of all the project players. 

Litigation 

• Claims and litigation are limited through proper risk allocation 
and assignment of responsibilities. 

• Method accommodates multi-parameter bidding schemes 
which allow for award based on factors other than price. 

• Contractual relationship between the Owner and Design/Build 
entity is significantly simplified 

• Owner is insulated from liability for design errors and 
omissions. Although the Design/Build contractor assumes 
responsibility, he is empowered with the ability to manage 
them directly. 


Table 2,2 Design/Build Disadvantages (Roth 1995) 


Area of Impact 

Design/Build Disadvantages 

Time 

• Design/Build contracts may take longer to award because of 
the complexity of the award process. 

• Design/Build process is more dynamic, requiring increased 
team and administrative participation. 

Cost 

• Cost of responding to IFB and developing proposal can be 
extremely expensive. This tends to limit competition and 
eliminate small firms. 

• Bonding costs for A/E and Contractor can be up to 50% 
higher. 

• Proposal cost is a sunk-cost, recovered only if contractor is 
awarded contract. 

• Modifications made after award can be extremely expensive 
if not made in a timely manner. 

• Increased responsibility of the Design/Build Contractor 
carries increased risk, therefore, he may increase his bid 
price for contingencies. 
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Table 2.2 Design/Build Disadvantages (Cont.) (Roth 1995) 


Area of Impact 

Design/Build Disadvantages 

Coordination 

• A/E’s direct link of communication with owner is removed. 

• A/E’s first allegiance is to the contractor not the owner. A/E’s 
feel their fiduciary role is changed. 

• Project scope must be defined extremely early in the process. 

• Process can be a real risk for unsophisticated owners not 
familiar with their administration. 

• Knowledgeable in-house staff must closely monitor project. 

• Importance of selecting an excellent project team is increased. 

• Inexperienced Subcontractors dislike the uncertainty of the 
process. 

Legal 

• Design/Build contracts are prohibited in some states. 

• Litigation may develop if the scope of work defined in the IFB 
is not absolutely clear. 


Although the Design/Build topic is very interesting and seems to be gaining 
momentum as a contracting vehicle, the intent of this study is to focus on the 
contracting of A/E firms that ultimately provide the drawings and specifications. 
The purpose of quickly reviewing the Design/Build issue is to show that there is a 
move in both the private and public sector to stray away from the conventional 
Design/Bid/Build is occurring. With the construction industry becoming more 
competitive and international companies entering the U.S. market, owners and 
construction companies in the private sector are looking for better ways of doing 
business, or areas that will save money. 

Apparently, there are pressures being placed on A/E firms by private 
owners to consider price as part of the selection method. This is contradictory to 
the Brooks Act. Has the private industry discovered a way to save money on 
design costs without impacting the quality of construction? There is probably a 
yes or no answer. There are various degrees of difficulty of construction projects. 












The bottom line is there has never been a valid study to prove or disprove the 
value of the Brooks Act. 

Thirty years ago when there was plenty of military construction dollars, the 
main focus with respect to A/E firms was not to sacrifice any design quality for the 
sake of saving a few dollars. Elaborate design aids such as CAD were not widely 
used. Technology with computers and information flow are now making 
tremendous advances and allowing many more competitors into the market. With 
the aid of technology, many simple projects can be reproduced very quickly for a 
fraction of the previous costs. Engineering is a very specialized skill, but has 
technology and education allowed the world to take a step up and change the way 
projects are designed? 

Regardless of the situation, the consensus of industry participants is that 
good design is essential to the success of a project. Not only is it true in the 
United States but countries such as Japan place a great emphasis in the design 
phase. A project that is well designed will experience less changes and ultimately 
give the owner a safer and more maintenance free facility. Since 85 percent of a 
facility’s costs are experienced after the building is completed (Bell et al. 1990), 
why are some private owners pressuring A/E firms into a price-driven qualification 
process? 

There are many issues and questions that have been raised. The answers 
are not straight forward, but before attempting to even address the issue, a 
statistical analysis comparing QBS projects to non-QBS projects should be 
conducted. Is the quality of some non-QBS designed facilities the same as QBS 
ones? If so, then the construction industry can probably save millions of dollars 



annually. Obviously, projects that are complex in nature and are on the cutting 
edge of technology will most likely require the best A/E firms. If there is a 
significant difference between the quality of non-QBS projects and QBS projects, 
then the Brooks Act needs to be reinforced as the best way to do business. 

Before gathering data to prove or disprove the hypothesis, the current 
laws, regulations and policies need to be reviewed. The ultimate goal of NAVFAC 
is service to the Naval Fleet through innovative ways of providing better facilities 
at reduced cost. However, the avenue chosen to give the best service has 
boundaries, therefore, it is important to examine regulations and processes that are 
currently being used in NAVFAC. 

2.2 Laws, Regulations & Policy 

There are basically two procurement options to obtain A/E services within 
NAVFAC as illustrated in Fugure 2.1. Each of these avenues contain eight distinct 
steps starting with the original procurement strategy to the ultimate award of the 
contract. Essentially, the A/E services can be obtained by separate design and 
construction which falls into the traditional method. The second approach is to 
obtain the A/E services through the Design/Build route. For the purpose of this 
study, separate design and construction will be the focus. In Design/Build, the 
design fees and construction costs are included in the eight step process. In the 
traditional sealed bid approach, the A/E has been chosen and is normally included 
in the procurement strategy of the construction contract. (GSA 1994) 



Procurement 

Strategy 

Specification 

Design 

Contracting 

Method 

Contracting 

Method 

Variance 

Solicitation 

Type 

Solicitation 

Response 

Contract Type 

Award Basis 

Separate and 
Construct 

Prescriptive 
Specification { 

Sealed 
Bidding _ 

Traditional 

Invitation for 
Bid 

Sealed 
Bidding _j 


Low 

Responsible 

Responsive 

Bidder 

Design Build 

/ 

Performance 

Specification 

\ 

/ 

* Sealed Bid v 
/ \ 

1 Newport 1 
Design Build 

Two Step 
Sealed Bid 1 

Invitation 1 
for bid 

Request for 
Proposal/ — 1 
Invitation for 
Bid 

Traditional \ 

/ 

Traditional 

Si ^ 

* Fixed Price 
r 

Low 

Responsible 

Responsive 

Bidder 



\ 

Negotiation^ 

Source 
f Selection 

, / 
Unweighted 
Negotiation 

Request for_, 

Proposal 

/ 

Traditional 

\ 

Cost 

f Reimburse 

Fixed Price^ 

Successful 
^ Proposer 


Figure 2.1 Procurement Options 



















Prior to examining the details of the laws and regulations, a cursory review 
of the history of the major events leading to the Brooks Act will explain the 
philosophy or the intent of the act which congress passed in 1972. In 1947 the 
Armed Services Procurement Act gave the government the authority to contract. 
Naturally, the government was contracting prior to these dates but this Act gave 
official direction. Since approximately 1939, A/E services were obtained based on 
qualifications rather than price. The philosophy is quite simple and clear. When a 
professional registered skill is required, the most natural approach people take is to 
attempt to get the most qualified person within reason. Good examples are 
doctors and lawyers. Architects and engineers generally fall into a similar 
category, especially with the complex design of buildings, utilities and bridges. 
Proper design is crucial for the safety to the public. 

Based on this premise, A/E services were obtained with that type of 
philosophy. Essentially, the government was conducting business in this fashion 
for 30 years. Then in 1967 the Comptroller General recommended that price 
should be considered in the selection of A/E services and that the six percent 
limitation on A/E fees should be repealed. In response to the Comptroller’s 
recommendation, H. R. 12807 clarifies procedures for the selection of A/E firms. 
In summary, the bill kept the current practice of using qualifications for selection, 
and the six percent limitation fee maintained. (NAVFACCONTRACEN 1996) 

The six percent fee refers to the preparation of designs, plans, drawings, 
and specifications and can be no more than six percent of the estimated 
construction cost of the project. The six percent fee does not include engineering 
services such as field investigations. In some cases, especially in the 
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environmental arena, the cost of engineering services may exceed the cost of 
performance. 

In 1972, the Brooks Act was passed. The law spelled out specific 
procedures for A/E contracting. Congress declared its policy for the Federal 
Government to publicly announce all requirements for architectural and 
engineering services, and to negotiate contracts for architectural and engineering 
services on the basis of demonstrated competence and qualification for the type of 
professional services required and at fair and reasonable prices. (Brooks Act 1972) 
The law is very logical and well thought out. Since engineering is a specialized 
field, it is prudent to hire the most qualified firm as long as the prices are fair and 
reasonable. This makes even more sense if an A/E firm is selected to design a 
complex project such as the launching pad for the space shuttle. Not only is that 
system very complex, but safety is another concern for the people who will be in 
the shuttle. That situation is clear cut, i.e. to select the best. 

But what about circumstances such as the design of a perimeter fence, a 
softball field, or an office building? Does it make sense to go through a rigorous 
selection process to choose the best A/E firm? In the future, will owners be able 
to purchase off the shelf designs for these types of facilities at low bid. A solid 
approach to analyzing the situation is to link the method of selection of A/E firms 
with factors that relate to project success. This is a very objective and unbiased 
approach. 

Over the past 10 years, Congress has been moving towards improving the 
contracting process. In April 1985, the Competition in Contracting Act was 
passed which simply amended the Armed Services Procurement Act by 



substituting sealed bids for formal advertising and replaced 17 exceptions with 
seven. The most recent law was passed FAS A in 1995. The purpose of the law 
was to speed up the government’s contracting process especially with small A/E 
contracts. Essentially, this raises the short selection process (SSP) threshold from 
$25,000 to $100,000. 

The SSP only requires interviews over the phone with a minimum of 3 
firms. There is no longer a 30 day requirement to wait for a response from A/E 
firms to receive their proposals. The Contracting officer can designate a 
reasonable period. Additionally, the A/E firms are no longer required to turn in a 
Standard Form 255. This form describes the A/E’s relative experience and 
provides a formal proposal tailored to a specific project. The only form required is 
the Standard Form 254 which is the qualifications of the A/E company and only 
needs to be updated on an annual basis. For A/E services that exceed $100,000, a 
Standard Form 255 must be completed on each contract. 

For A/E services less than $100,000, a contract can be awarded in one of 
two methods. The first method is the Chairman of the pre-approved standing 
board can select an A/E firm based on a minimum of three phone interviews and a 
review of the Standard Form 254 submitted or ones that are held on file. Then the 
Chairman of the Board gives his written recommendation to the Contracting 
Officer for signature. The other method is a standing board of three members 
appointed by the Engineering Field Division Commanding Officer who can directly 
approve the selection of an A/E firm. However, the board must then write a report 
on why a the A/E firm was selected. 
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Every field office varies with respect to organizational structure and 
mission. Obviously, to set one specific guideline to meet all the objectives would 
essentially be impossible. Therefore, the Department of Defense (DoD) falls into 
the broad category of the Federal Acquisition Regulations (FAR) and the DoD 
FAR supplement. These are the primary procurement regulations for the DoD. 
The Navy Acquisition Procedures Supplement (NAPS) provides additional 
guidance in the area of Navy contracting. NAVFAC has further defined the 
guidance to include specific authority, responsibilities and internal procedures. 
The NAVFAC Publication P-68 is the source of information for the delegation of 
authorities for the execution and administration of A/E contracts. Finally, the 
Engineering Field Divisions (EFD) set more refined guidance tailored to the 
specific organizational needs and goals. (NAVFACCONTRACEN 1996) 

2.3 Typical NAVFAC Acquisition Process 

FASA has enabled the government to expedite many contract actions. The 
trend is to provide the services to the customers without jeopardizing competition. 
Depending on the situation and the amount of A/E contracting, the typical 
acquisition process for A/E contracts greater than $100,000 is shown in Table 2.3. 
(NAVFACCONTRACEN 1996) 



Table 2.3 NAVFAC A/E Acquisition Process. (NAVFACCONTl 

RACEN 1996) 

Process 

Action 

Time 

Requirement identification 

User 

Varies 

Acquisition planning 

EFDPM 


Advertisement 

Contracts 

2 Months 

Qualification statements received 

Evaluation Board 

1 Month 

Slate/preselection board meeting 

Evaluation Board 

2 Weeks 

Slate/preselsction board report 

Chairman of the Board 

1 Week 

Request for Proposal 

Contracts 

1 Week 

Government estimate preparation 

EFDPM 

2 Weeks 

Receipt/review of proposal 

A/E Firm 

2 Weeks 

DCAA Audit (over $500K) 

Contracts 

■ESES59I 

Prenegotiation objectives established 

Contracts/Tech Rep 

2 Weeks 

Negotiation 

Contracts/Tech Rep 


Post Negotiation Memorandum 

Contracts 

1 Week 

Contract Award 

Contracts 

1 Week 

Contract Administration 

Contracts/ROICC 

Varies 


2.4 Length of Acquisition Process 

In 1989, NAVFAC conducted a study to determine how long it took to 
obtain A/E services. The results showed that from the time a project was 
authorized until the contract was signed, 330 days had elapsed. This was 
unacceptable to NAVFAC. Private industry would go out of business waiting 
nearly a year to select their design firms. As a result of the study conducted in 
1989, NAVFAC made some recommendations which led to (NAVFACENGCOM 





































• Less time required to award an A/E contract. 

• More responsive to customer needs. 

• Better customer satisfaction. 

• Increased quality of A/E selection and supporting document 

• Reduced paperwork. 

• Enhanced teamwork and ownership of the outcome. 

• More efficient manpower utilization 

As part of this study, a brief survey was conducted by telephone 
interviewing Engineering Field Divisions and Engineering Field Activities, 
(EFD/EFA) within NAVFAC. The interviews conducted include the following 
four Commands: EFA Northwest, EFA Chesapeake, EFD Southwest and EFD 
Pacific. The result of survey indicates that, the average procurement time has been 
reduced to four months, a seven month improvement in the acquisition process. 
This supports information provided in the curriculum taught at the Naval Facilities 
Engineering Command Contracts Training Center which states that the normal 
procedure for contracting an A/E firm should take 117 days for a standard project. 
Table 2.4 shows the time frames associated with different project priorities. 

The time frames for the four other methods vary from 32 to 74 days to 
award a contract. Track B, Fast Track, essentially, goes through the same process 
as the standard, but involves more senior management interest and the contract is 
hand carried through the process. Track C, Generic Slate, are routine projects that 
are less than $10 million. Basically, a slate of three firms are chosen from a pre¬ 
advanced list by region. The updating of the list is done twice a year. A standing 
slate board then chooses from 3 interested firms. Essentially, the lengthy process 
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of formal slate/select boards is eliminated. Track D, Urgent and Compelling, is 
used if the project is considered mission essential, an operational necessity, or the 
customer certifies urgency. In any case, a Justification and Authorization (J & A) 
must be signed for this approach. Finally, Track E, the Indefinite Quantity, (IDQ) 
contracts are used as needed by type of contract. Normally a minimum and 
maximum amount are stated on the contract. One large contract is awarded to an 
A/E firm and as design services are needed within scope of the originally contract, 
a delivery order is written that specifies the requirements and specific scope of 
work to be accomplished. (NAVFACCONTRACEN 1996) 


Table 2,4 Priority System. (NAVFACCONTRACEN 1996) 


Track 

HHNHBSSSSBHIHii 

Time 

A 

Standard Process 

117 

B 

Fast Track 

74 

C 

Ready/Generic Slate 

37 

D 

Urgent and Compelling 

32 

E 

Indefinite Quantity 

51 


While the study conducted in 1989 led to the initiative as mentioned 
previously, NAVFAC also gave the EFD/EFA/PWC’s flexibility to adapt work 
processes to meet the user’s needs while maintaining their own organizational 
goals. 

Even more important today with the closing of bases and restructuring, the 
Navy continues to look at ways to reduce manpower while improving capability. 
Technology will play the primaiy role in being able to accomplish this task. That is 
why it is important to periodically address questions such as does it make sense to 
spend valuable resources and money trying to choose the most qualified A/E firm 
for every project? FASA has resolved the time length issues for small contracts. 




















Why not review current projects and base decisions on the successes and failures 
of projects rather than a set policy based on anecdotal information? The A/E 
selection procedure really has not been reviewed since the early 1970’s. The same 
philosophy holds true for Design/Build projects. Now that the Navy has been 
using design build for over five years, it is time to conduct statistical study to 
analyze project data to determine how much benefit or impact the Design/Build 
process has versus the Design/Bid/Build. 

2.5 Phase I Background 

The following discussion from the Phase I Background through the next 
two Chapters is taken from the Phase I report A/E Service Selection Method 
which was generated by the Department of Civil Engineering at the University of 
Texas at Austin (Vanden Bosch et al. 1996). 

Key participants within the construction industry report a growing new 
trend in design firm selection from selection based solely on the qualifications of 
the design firm towards selection based solely on price. However, selection based 
solely on price alarms many design firms and professional organizations, who feel 
that such a focus will substantially degrade the quality of the design service. This 
will in turn result in completed projects that do not fully meet client needs and 
expectations. 

Reoccurring attempts have been made both to weaken and to strengthen 
the Brook’s Act. To date, the act has survived intact, but is under increasing 
attack. The Justice Department has required professional societies to remove 
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ethics clauses in their by-laws that prevented the competitive bidding of services on 
the grounds that such clauses violated anti-trust laws. Thus inherent tensions 
between pricing pressure and qualifications pressure for design services are likely 
to continue. The Brook’s Act is not without its critics, who charge that the act is 
inherently biased, discriminatory, and restricts competition. These criticisms are 
further explored in Chapter 3. 

The current arguments both for and against QBS are based on anecdotal 
data. The problem with anecdotal examples is that there is no test to determine if 
they are extreme examples of infrequently occurring events, or to what degree the 
examples represent the central tendencies of the general population. Because of 
this, anecdotal examples cannot be used to establish population trends or even to 
prove a point. Therefore, a statistical study needs to be performed to evaluate the 
efficacy of QBS. 

2.6 Objectives of Phase I 

Phase I set up the framework for this study. The primary objectives of 
Phase I were to: 

1) Develop data gathering procedures and tools for further study, 

2) Address sample size and availability of data for further study, 

3) Conduct reviews of the proposed methodology in order to refine 
gathering procedures and tools, 

4) Make necessary refinements in the procedures and tools, and 

5) Communicate all findings in a report. 

2.7 Scope of Phase I 

The scope of Phase I was to develop statistically sound survey tools for 
gathering relevant project information linking a project’s performance and the 
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method of selection of its design service provider. The scope of research is 
therefore to: 

1. Build on prior studies 

In any study involving substantial issues and possibly affecting policy 
and/or legislation, it is incumbent on the research team to extensively search 
existing literature for similar or related studies, and, if such studies exist, build on 
their results. Phase I of this study includes such an effort. The results of the 
literature review are presented in Chapter 3. 

2. Develop/Modify tools 

Every statistical study is unique. The researchers, therefore, must develop 
a survey instrument and approach that meets the needs of this study. The survey 
instrument from phase I has been refined and is given in appendix A. 

3. Address statistical issues 

There are many issues associated with statistical analysis that can affect the 
usability of data.. Failure to account for these issues could damage the analysis 
effort. These issues are briefly discussed at the end of Chapter 3 and further 
elaborated in Chapter 4. 

4. Get preliminary industry reaction to methodology and tools 

The proposed exploratory survey methodology and data gathering 
approach were evaluated by independent individuals knowledgeable in the A/E/C 
industry. Improvements were made based on their suggestions. Preliminary 
feedback on the draft methodology and survey tools is presented in Chapter 3. 




2.8 Methodology of Phase I 

The methodology employed in Phase I is depicted in Figure 2.2 and has 
been completed. This study is the pilot test of the survey tools developed in 
Phase I. 


Figure 2.2 Methodology of Phase I 





Chapter 3: Preliminary Investigation 

3.1 Literature Summary 

An extensive literature search was performed during Phase I, and an 
annotated bibliography of the relevant literature is presented in the Phase I report. 
Findings of the literature review are summarized below. (Vanden Bosch et al. 
1996) 


The Brook’s Act 

As previously discussed, the Brook’s Act controls how federal agencies 
select design firms for federal projects. The act requires federal agencies to invite 
at least three architect or engineering firms to be considered for each selection 
process. These firms are then ranked in order of their qualifications alone. The 
highest ranked firm is selected. Pricing considerations are addressed in the Act by 
requiring the agency to negotiate only with the highest ranked firm, and if 
negotiations are successful, award to that firm. If the negotiations are 
unsuccessful, the agency terminates that selection and moves on to the next highest 
qualified firm. (FAR 36.6) 

Critics of the Brook’s Act complain that the act is inherently biased toward 
large, established firms, and discriminates against new, small, and minority firms. 
(Hampton 1994) Such criticisms have appeared in court cases contesting the 
selection of one design firm over another. Without disputing such allegations, the 
federal courts have nevertheless upheld the Brook’s Act, and it has withstood all 
such challenges. (485 Federal Supplement 1292; B-201395.2 1982; B-218404.2; 



B-218474) Another concern is that even though the Federal Acquisition 
Regulations (FAR) declares that the selection process promulgated by the Brook’s 
Act must meet the requirements of “fair and open competition” by law, in fact it is 
restricted competition with sole-source negotiations, two terms which experienced 
public procurement officials usually associate with exorbitantly higher prices. 
(FAR 36.6) 

To counter against the assaults on the Brook’s Act, the major professional 
societies have formed an organization to review and study the issue of how best to 
defend the Brook’s act. That organization is the Committee on Federal 
Procurement of Architecture and Engineering Services (COFPAES). COFPAES’ 
interest, though primarily federal, extends to the state and local levels. In addition 
to COFPAES, a recent report from Massachusetts presents anecdotal information 
supporting the Brook’s Act’s use. (AIA 1996; Massachusetts Taxpayer’s 
Foundation 1995) 

There are many processes in use for selecting the design service provider. 
Perhaps the most common classification is a modification of the classification 
system used by the Professional Services Management Journal (PSMJ) in their 
annual design fee survey. (PSMJ 1996) The modification of the PSMJ 
classification rather neatly proceeds from pure price-based selection, to pure 
qualifications based selection and was given earlier in Table 1.1. 

Price-Based Selection Trends 

The trend towards price-dominated selection of A/E services appears to be 
leveling off. There is some indication that the practice is abating as owners and 



others learn that the anticipated costs savings do not occur in the long run. 
(Architectural Record 1994; Hathway 1995; Ichniowski etal. 1995; PSMJ 1996) 
Price-only selection comes from private owners, other service providers, 
construction contractors, and public agencies at the federal, state and local levels. 
Pricing considerations in the selection of a design service provider are significant in 
the public sector, as well as the private sector. By law, some states require the 
design service provider to bid for public projects. Many other government and 
quasi-government agencies, districts, boards, and other public service providers 
appear to practice selection of the project designer based only on price, regardless 
of laws apparently to the contrary. Surprisingly, A/E firms subcontracting work to 
smaller consulting firms often do so based primarily on price. (Angelo 1995; 
Schriener 1995; Massachusetts Taxpayer’s Foundation 1995; PSMJ 1996) 

Public sector price pressures driving selection of design service may be 
attributed in part to the Federal Acquisition Regulations (FAR) Part 6 - 
Competition Requirements, and in particular to Subpart 6.1 - Full and Open 
Competition, where paragraph 6.101 Policy establishes full and open competition 
as the normal policy of the land. All other procedures are limited exceptions 
usually requiring a written justification and higher authority approval, unless a 
statutory exception (such as the Brook’s Act) provides otherwise. Moreover, 
Subpart 6.5 — Competition Advocate, requires each agency to appoint a 
competition advocate to actively promote full and open competition. Finally, the 
Brook’s Act itself appears to be mandatory only when the contracted services 
require a licensed professional A/E to perform them. Those A/E services not 
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requiring a professional license and signature may not be mandatory. (Lunch 1994; 
USCA 40; B-201395.2 1982; FAR Part 6) 

The public policies embodied in the FAR regarding competition are rooted 
in a pronounced desire by taxpayers to minimize their tax burden. Taxes are the 
pre-dominate source of government revenues for public projects. One of the 
consequences of this is reflected in public policies at all levels of government to 
obtain goods and services for public use at the lowest reasonable price. The 
predominate method adopted by nearly all public procurement offices to minimize 
the cost of such goods and services is to define the requirements, and publicly 
invite all interested parties to submit their price (bids) for providing the goods or 
services. (Lunch 1991; Korman etal 1995; ENR 1991; DeFraites 1989) 

Private sector price pressures driving selection of design services may be 
attributed to the perception that design services are a commodity, caused in part by 
design firms limiting their liability to the owner. The extensive use of standard 
form design contracts also adds to this view. The perception that there is little 
difference between A/E firms is pervasive. (Architectual Record 1994; Korman 
etal. 1995; Grogan 1995; PSMJ 1996) 

Project Performance 

In a series of meetings held in Fall 1995, the research team developed a list 
of project outcome and project process performance criteria based on team 
expertise, and significant prior research work performed by the Construction 
Industry Institute. (CII et al. 1986-1994) The project performance categories 
include: 




• Total installed cost (TIC) cost performance 

• TIC cost performance relative to industry benchmark 

• Unit construction cost 

• Cost of design quality relative to TIC 

• Cost of construction quality relative to TIC 

• $ amount of claims or litigation relative to TIC 

• Overall schedule performance relative to owner goals 

• Overall schedule performance relative to industry benchmark 

• Jobsite safety performance relative to goal 

• Jobsite safety performance relative to industry benchmark 

• Level of customer (owner) satisfaction 

• Plant/facility output performance relative to owner requirements 

• Safety of facility operations and maintenance 

• Aesthetics of facility 

• Cost efficiency of facility operations 

• Cost efficiency of facility maintenance 

• Job profit of A/E 

• Job profit of contractor 

• Satisfied project personnel 

• Accessibility to new service providers 

• Ease of overall project delivery from owner’s perspective 

• Amount of required owner front-end costs 

• Fair system of selection with integrity and minimal corruption, 
abuse, or protests 

• Ease and uniformity in execution of selection process 

These project outcome and project process performance categories were 
used to develop the survey instruments as described in Chapter 4. Issues of 
diversity, complexity, and difficulty in measurement were addressed by the 
research team as well. 

There are many influences to project success. One of the most important 
influences is in the design stage of the project, where small changes in the design 
process may have enormous cost consequences in construction or operations. 
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Statistical Studies 

The literature review revealed no rigorous statistical studies concerning 
qualifications-based selection of A/E services. The study will therefore be 
exploratory in nature and should include multiple sources of data and data 
gathering methods as discussed in Chapter 4. 

3.2 Overview of Statistical Procedures 

Several methods exist for statistical analysis of data. All generally 
acceptable methods today are founded in probability theory and a rigorous and a 
carefully executed data collection process. (Knoke etal. 1982; Babbie 1990) To 
roughly frame the available analysis techniques, two broad techniques are used 
below: means and variance testing, and analysis of variance. The Phase I report 
addressed these techniques in detail (Vanden Bosch et al. 1996) 

Of the many thousands of projects that could be examined, there is time 
and money for only a limited sampling of the projects that exist. Generally, 
completed projects are necessary to measure overall project performance, so all 
uncompleted projects will be excluded. Much of the data required for this study is 
not likely to be kept in a single file, or even in a single location. In addition, much 
of the data is probably resident only in the minds of the key project participants. 
Therefore, the availability of data, and the availability of knowledgeable persons, is 
a requirement. Projects completed more than five years ago will be excluded on 
the basis that the time lag for personal recall is too great to be reliable. 



Projects too small or too large in size may disproportionately distort the 
analysis results. Therefore projects less than $2 million and more than $60 million 
are expected to be excluded from the study. 

The diversity of projects in the universe of all construction projects is too 
great to include all types of projects. It was proposed to limit the project sectors 
to buildings, horizontal construction, and light industrial. 

The building sector consists of general buildings and low-rise structures 
such as jails, prisons, correctional facilities, schools, maintenance facilities, office 
buildings, retail stores, but not high-rise buildings. 

The horizontal construction sector consists of roads and highways, earth 
dams and levees, flood control structures, irrigation, power transmission systems, 
water distribution systems, sewage collection systems, and grading, excavation or 
landscaping. 

The light industrial sector consists of power plants, water treatment plants, 
and sewage treatment plants. 

3.3 Preliminary Interview Feedback 

The developed survey tools and proposed methodology were reviewed 
with knowledgeable industry officials to provide a reality check on their direction, 
applicability, and completeness. In addition, the interviews uncovered areas that 
require further attention. These recommendations were incorporated into the 
survey design and methodology. (Vanden Bosch 1996) 




3.4 Pretest 


This thesis addresses the need to further test the survey questionnaires on a 
small, select sample in order to get detailed feedback. The next chapters give 
detail of this effort along with initial results of this small sample. In addition, a 
brief critique of the Phase I survey instruments is given as well. 



Chapter 4: Methodology 


4.1 Hypothesis 

In statistical analysis, a hypothesis is a statement of the intent of the 
statistical analysis, i.e., what it is the study is attempting to establish. For this 
study the hypothesis is, in its simplest form, that project performance is related to 
the method of selection of the designer. More specifically, project performance is 
negatively related to pricing pressures exerted on the design service provider 
during his/her selection process. A hypothesis cannot be directly proven, so its 
opposite, or null position is disproved. (Knoke etal. 1982; Babbie 1990) The null 
hypothesis for this study is that there is no significant relation between project 
performance and the method of selection of the design service provider. 



4.2 Methodology 


This section presents the steps and data collection techniques that were 
executed and discusses statistical analysis issues affecting the usability of the study 
results. Figure 4.1 shows the methodology for the proposed study. The work 
accomplished in this thesis has addressed the methodology on a small scale in 
order to pilot test the survey approach. 


Figure 4.1 Methodology 
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4.3 Data Gathering Procedures 

The sampling frame was developed from available sources. Once the 
sample projects were identified, key individuals for each project in the sample were 
contacted. Personal contact, either by telephone or face-to-face, was made with 
each key project individual. 

Once the questionnaires were delivered, follow-up interviews were made as 
required to address questions, or to maintain and induce enthusiasm in the 
respondents to complete the questionnaires, thus insuring a reasonable sample 
response rate. Data quality was monitored as the data were received, and follow¬ 
up contacts made to resolve unclear and missing data. 

4.4 Data Analysis Procedures 

Meaningful data analysis is largely dependent on receiving all or most of 
the sample data. While the major focus of this study was project performance and 
the A/E selection method, there were other factors that affected project 
performance. Not every factor below affected this pilot study, but are included 
on the list to show the factors that were taken into account: 

(1) Externally driven factors, such as 

• competitive market conditions 

• local regulatory and permit requirements 

• labor strikes 

• material availability 

• abnormal weather 

• civil unrest, war, and insurrection 

• acts of God 
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(2) Owner controlled events, such as 

• pre-project planning 

• contracting strategy 

• design reviews 

• constructability program 

• construction manager 

• time constraints 

• use of incentives 

• financing or funding constraints 

• partnering 


(3) Project specific factors, such as 

• Remoteness of the site 

• complexity of the project 

• uniqueness of the project, e.g.: 

• prototype 

• first of a series 

• new process or technique 


(4) Constructor controlled events, such as 

• experience 

• proclivity to claim 

• attitude 

• safety practices 

Project success is a combination of cost, schedule, quality and owner 
satisfaction. All of the above factors can independently affect any and all of these 
project success measures. Therefore it is necessary to include, or account for, the 
influence of these factors in the data. (Knoke etal. 1982; Babbie 1990) Once all 
these other influencing factors were controlled for in data collection, the resulting 
data was analyzed regarding project performance and A/E selection method. 



At this point in the analysis, the sub-sample groupings were tested to see if 
they were significant, i.e., if they were statistically different from each other. If the 
sub-sample groupings were significant, then two questions remain: 

1) “Do the project success factors co-vary with the A/E selection 
methods?”, and if so, 

2) “How strong is the co-variance relationship?” 



4 


4 


4 


4 


m 


4 




4.5 Data Gathering Tools 

Based on the project performance categories presented in Chapter 3, a 
matrix of questions and requirements was developed as shown in Table 4.1 
corresponding to the questionnaire presented in Appendix A. 


TABLE 4.1 Performance Criteria/Questionnaire Matrix 


Performance Criteria 

m 

Survey Question(s) 

121 

OUTPUTS: 


1. TIC Cost Performance 

2.2.1 

2. TIC cost performance relative to industry 
benchmark 

2.2.1 and 2.1.6 (make our own 
sample comparison) 

3. Unit construction cost 

2.1.6 

4. Cost of design quality relative to TIC 

2.2.1 and changes question(possibly) 

5. Cost of construction quality relative to TIC 

2.2.1 and changes question 
(possibly) (4. and 5. are difficult and 
probably understated) 

6. $ amount of claims or litigation relative to TIC 

2.6.1 and 2.2.1 

7. Overall schedule performance relative to owner 
goals 

2.3.1-6 

8. Overall schedule performance relative to industry 
benchmark 

2.1.6 and 2.3.1-6 (make our own 
sample comparison) 

9. Jobsite safety performance relative to goal 


10. Jobsite safety performance relative to industry 
benchmark 

N.A. 

11. Level of customer (owner) satisfaction 

2.7.1.a; 2.7.8 j 

12. Plant/facility output performance relative to 
owner requirements 

2.5.1; 2.5.2 

13. Safety of facility 0 & M 

2.7.6.a-c 

14. Aesthetics of facility 

2.7.3.C 

15. Cost efficiency of facility operations 

2.7.5a-c 

16. Cost efficiency of facility maintenance 

2.7.6.a-c 

17. Job profit of A/E 

N.A. 

18. Job profit of contractor 

N.A. 

19. Satisfied project personnel 

N.A. 

20. Accessibility to new service providers 

N.A. 

21. Ease of overall project delivery from owner's 
perspective 

N.A. 

22. Amount of required owner front-end costs 

N.A. 

23. Fair system of selection with integrity and 
minimal corruption, abuse, or protests 

2.6.1; 2.6.2 

24. Ease & uniformity in execution of selection 
process 

N.A. 
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Using this matrix, the project performance questionnaires were developed. 
Three variations of the project performance questionnaire were developed, 
however, only the one for the building sector was used in this pilot study. 

Column 2 of Table 4.1 shows the question number in the project 
performance questionnaire. Issues in Table 4.1 without corresponding questions 
are shown by “N.A.” in column 2 of the table. Such issues are not addressed 
because the research team felt they were too hard to obtain data, or were too 
sensitive. 

In addition to the performance measurement questions generated from 
Table 4.1, questions to control for the other factors that affect project success, 
discussed in the antecedent Data Analysis Procedures section, are included in the 
project performance questionnaire under question 2.1.7. These questions are 
designed to allow the isolation of the effects of the A/E selection method on 
project success factors in the data analysis. 

Finally, questions are included in the questionnaires to probe respondents in 
terms of project type, designer, background information on the project, 
identification of key personnel, participating firms in the project, etc. These types 
of questions are at the beginning of the questionnaires. See appendix A for the 
final copy of the project performance questionnaire. 
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4.6 Service provider questionnaire 

The A/E point of contact (POC) was included in the project performance 
questionnaire. Once this information was received, a separate service provider 
questionnaire was sent to the A/E. 

The service provider questionnaire was developed in brainstorming 
sessions by the research team incorporating the results of the literature review. A 
separate questionnaire was developed because it became clear that the design 
service provider was best positioned to answer those questions, while the owner 
was best positioned to answer the project performance questionnaire. Projects 
were segregated into QBS, non-QBS, or a combination of the two based on 
responses to this questionnaire. See appendix B for the final copy of the service 
provider’s questionnaire. 



Chapter 5: Presentation and Data Analysis 

5.1 Characterization of Sample 

The pilot sample comes from a combination of two sources. The first 
source was generated by F. W. Dodge who provided 461 randomly selected 
building sector projects. These projects have been completed within the last year 
and are representative of America’s construction industry. The other source of 
projects came from locally selected projects in the Austin, Texas area based on 
personal contacts. From the 461 projects, nine projects were randomly selected 
within the state of Texas and three projects were chosen locally. The scope of the 
sample size was limited to Texas because of funding and timing constraints. 

This study will set the foundation for the next research phase which will be 
conducted on a much broader scale. The more comprehensive analysis will 
include projects encompassing the United States and will also include the 
horizontal and utilities sectors of construction. As discussed earlier, this study is a 
pilot study to determine the response rate and quality of questionnaire. Based on 
the feedback provided by the initial sample, the questionnaire will be modified. 

The specific types of projects chosen are schools, apartment, and office 
/retail buildings. The size of projects range from 70,000 to 320,000 gross square 
feet (GSF) and total installed cost of 4 to 25 million. The response rate of all the 
questionnaires sent out was 66 percent. As previously mentioned in the 
methodology, the owners were initially contacted by phone and ask if they were 
willing to participate. Once the questionnaires were received from the owners, 
the A/E firms were contacted and asked to fill out the service acquisition 
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questionnaire. There were a 100 percent response rate once the A/E firms knew 
that they were referred to by the owners. Table 5.1 characterizes the sample used 
in this study. 


Table 5,1 Sample Characterization 



SF 

Totals 

Location 


Source 

(D 

(2) 

(3) | 

(4) 

_(5)_ 

(6) 

P001 

120,000 

$10,676,000 

Austin, TX 


Local 

P002 

319,000 

$21,770,000 

Irving, TX 

Manufactung 


P003 

197,000 

$19,000,000 

Travis CA 

Retail 


P004 

62,957 

$4,104,725 

Laredo, TX 

Hotel/Motel 

HZ23333 

P005 

71,000 

$4,325,930 

Harris C TX 

Retail 

FW Dodge 

P006 

65,000 

$11,600,000 

Plano, TX 

School 

E333SU 

P007 

67,000 

$4,500,000 

Austin, TX 


Local 

P008 

78,000 

$6,070,658 

Austin, TX 

School 








AVG 

122,495 

$10,255,914 





5.2 Characterization of QBS and Non QBS 

The Brooks Act is very clear on the selection process for A/E firms. The 
selection of the firm has to be made based on qualifications alone, then 
negotiations occur with a goal of obtaining a fair and reasonable price not to 
exceed six percent to the most qualified firm. If the negotiations are unsuccessful, 
the negotiation process goes to the next most qualified firm and so on. The exact 
opposite of QBS would be selection based on price alone such as a sealed bid. A 
third category is a combination of the QBS and non-QBS. This can happen a 
number of ways, but typically, the owner will make his selection after reviewing 
pricing data. In essence, the owner is looking for the “Best Value”. Table 5.2 
below again summarizes the different categories that were addressed earlier. 


























































Table 5.2 A/E Selection Methods (Vanden Bosch et al. 1996) 



DESCRIPTION 

■5HKSI33H1 

(Price) 

Bid 

Price is the only selection determinate. 

Combination 

(.Pre-qualification, Price) 

Two 

envelope 

The first envelope contains qualifications only. All highly 
qualified firms are pre-qualified. Pre-qualified firms are invited 
to submit price proposals. The second envelope contains the 
price proposal. Award is based on best price. 

One 

(Best value, considering qualifications and price) 

envelope 

Firms simultaneously submit qualifications and price. Award 
may be based on some combination of price and qualifications. 

QBS 

The first envelope contains qualifications only. The one best 

Two 

qualified firm is selected and asked to submit a price proposal. 

envelope 

Price is negotiated with the one best qualified firm only. 

Sole source 

(Reputation is the selection determinant) 

The firm is selected based on their reputation and/or familiarity 
with client’s project. Price may or may not be negotiated. 


From this pilot study, only two categories were observed. The two are 
“QBS” and “Combination” based on responses from the service provider 
questionnaires. Table 5.3 separates these two types. 











5.3 QBS/non-QBS Characterization 



QBS, Q/Combination, C 

(2) 

P001 

Combination 

P002 

Combination 

P005 

Combination 

P006 

Combination 

P003 

QBS 

P004 

QBS 

P007 

QBS 

P008 

QBS 


None of the projects from the sample were based soley on price. In each 
case some degree of qualifications was a factor. The key to separating between 
“QBS” and “Combination” was question number three in the Service Acquisition 
Quesionnaire. The bottom line in separating the two was if the A/E firm was 
selected before negotiations took place, then the project was catogorized as QBS. 

Selection of these sample projects was not done on a purely random basis. 
The projects from the Texas area were targeted because of the time constraint of 
this effort. However, there were no measures taken to ensure a 50 percent sub¬ 
samples existed between QBS and non-QBS projects. Of the projects selected 
from Texas half were identified as QBS and the other half as a combination or 
“Best Value” which is explained in Table 5.2. From these results, the conclusion 























is that price pressures are probably being exerted by owners on A/E firms on a 
large number of projects in today’s construction industry. 

5.3 Analysis of Data 

There are numerous ways to measure project success. For the purpose of 
this study. The factors will be broken into cost, schedule, changes, positive and 
negative factors, subjective design ranking and overall project success ratings by 
the owners. In terms of cost, the area of concentration will be percentage of 
design cost and various cost per square foot comparisons. Evaluation of 
schedule will include actual time and also the subjective opinions of the owners. 
Changes will be presented as a percentage of actual construction cost. Finally, the 
positive and negative factors will be grouped in one table to summarize the 
owner’s overall ranking of the success of the project. All data presented will 
segregate the QBS and non-QBS projects. 

5.4 Design Cost 

The first analysis is to look at the cost of design. Remember, the Brooks 
Act allows no more than 6 percent of estimated construction cost for Federal 
projects. This provides a check on whether design fees exceed the six percent 
statutory limit and also gives a relative comparison between QBS and non-QBS. 
Columns (3) and (4) in Table 5.4 represent the cost of design and construction at 
the completion of the project. Column (5) is the percentage of design with respect 
to column (4). 
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Table 5 4 Design Cost of Sample Projects 


EZEIS3I 

Q/N 

Design, D $ 

Construction, C $ 

% D 

■SB 

(2) 

(3) 

(4) 

ESI 

P001 

C 

$300,000 

$10,676,000 

2.81 

P002 

C 

$1,400,000 

$21,770,000 

6.43 

P005 

c 

$126,000 

$4,325,930 

2.91 

P006 

c 

$610,000 

$11,600,000 

5.26 

C Avg. 


$609,000 

$12,092,983 

4.35 

P003 

Q 

$1,386,000 

$19,000,000 

7.29 

P004 

Q 

$167,748 

! $4,104,725 

4.09 

P007 

Q 


$4,500,000 


P008 

Q 

$299,773 

$6,070,658 

4.94 

Q Avg. 


$617,840 

$8,418,846 

5.44 


Focusing on column (5) in Table 5.4, two percent of the projects exceeded 
the 6 percent threshold set by the Brooks Act, one from each sample. The 
problem with evaluating design cost is the various definitions that exist in the 
construction industry. Design fees can include a multitude of services from site 
visits to drawings to assistance with contract administration. The Brooks Act 
allows no more than 6 percent of estimated cost of construction, however, this 
only includes basic services such as providing the specifications and drawings for 
the project. Other services such as site visits and modification support can be 
added on top of the 6 percent. Therefore, it is very important to ensure the 
design services for each project are specifically defined. 

As probably expected, the owners of the projects that applied price 
pressures on the A/E firms on the average paid 20 percent less for design services. 
The results are shown in column (4) of Table 5.5. 























































5.5 Cost/SF Analysis 


Cost per square foot is another essential success factor to analyze. 
Designers, developers and owners are continually thinking about the bottom line, 
cost per square foot. Most likely, the cost of the building or leasing space is 
estimated and leased on cost per square foot basis. Table 5.3 below shows several 
Cost/SF comparisons. Columns (2) and (3) are basically the design and 
constructions costs divided by the total square feet of the facility. 


Tab 

e 5.5 Cost/SF for the Sample 



Const. SF 

Q/N 

mm 

(2) 

■■ 

o 

P001 

$2.50 

$88.97 

C 

P002 

$4.39 

$68.24 

C 

P005 

! $1.77 

$60.93 

C 

P006 

$4.69 

$89.23 

C 

mm 

$3.34 

$76.84 


P003 

$7.04 

$96.45 

Q 

P004 

$2.66 

$65.20 

Q 

P007 

NA 

$67.16 

Q 

P008 

$3.84 

$77.83 

Q 

mm 

$3.39 

$76.66 



♦Average for P003, P004 and P008 


Analyzing columns (2) and (3) from Table 5.5, the owners that are 
applying price pressures are saving 1.5 percent on the design cost and spending .23 
percent more on the construction cost. Using the average size facility and 
construction cost from columns (2) and (3) of Table 5.2, average savings of the 
owners on design costs is $6,125 and the increase spending for construction is 



















































$22,049. The owners that went through a qualifications process saved more 
money than the non-QBS projects. The margin is essentially the same for a sample 
size of eight projects. On multi-million dollar projects, a difference of a few 
thousand dollars is not statistically significant. 

5.6 Schedule Analysis 

Completing a project on time is extremely important. In general, the less 
time the contractor stays on the project the more profitable he will be. This will 
also give the owner earlier access to the facility. Regardless which side of the 
project a firm is on, schedule is an important success factor to measure. In table 
5.6, columns (3) and (4) show planned and actual duration of the project in 
months. Column (5) is the percentage difference between columns (3) and (4). 


Tal 

Me 5.6 Schedule Analysis 


Q/C 

Planned 

Actual 

% Diff. 

mm 

131 

HH 

(4) 

(5) 

P001 

c 

8 

8 

0 

P002 

c 

12 

13 

+ 8 

P005 

c 

18 

18 

0 

P006 

c 

6 

6 

0 





+ 2 

P003 

Q 

12 

24 

+ 100 

P004 

Q 

8 

7 

-14 

P007 

Q 

16 

20 

+ 25 

P008 

Q 

24 

24 

0 




Q Avg: 

+ 28 


The non-QBS type projects rated much better in terms of completing on 
time. Project P003 seems to sway the results. The delays in that project were 
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mainly caused by a weather delay of 20 weeks and engineering and construction 
productivity delays of 18 weeks. Even if that data point is thrown out, the non- 
QBS projects would have still completed their projects 10 percent faster. This is 
opportunity cost lost by the owner and the contractor. Again, a larger sample size 
and a more specific breakdown of delays caused solely by design errors should be 
analyzed. 

5.7 Changes Analysis 

The majority of the contractors and owners prefer no changes because they 
disrupt the schedule and have a cascading effect on other aspects of the project. 
Changes occur on almost every project and the key is to minimize the changes that 
are controllable. In table 5.7, column (3) is the number of changes that occurred 
on the project. Column (4) represents the cost of the changes and column (5) is 
the percentage of changes with respect to the actual cost of construction. 


Tal 

jle 5.7 Changes Analysis for the Sample 


Q/C 


Changes ($) 


na 

■51 

(3) 

(4) 

(5) 

P001 

c 

25 

$ 1,400,000 

13.1% 

P002 

c 

134 

$ 2,000,000 

9.2% 

P005 

c 

26 

$ 1,751,036 

40.5% 

P006 

c 

0 

0 

0 




Avg: 

15.7% 

P003 

Q 

108 

$ 7,000,000 

36.8% 

P004 

Q 

4 

$ 111,993 

2.7% 

P007 

Q 

NA 

$ 20,000 

0.4% 

P008 

Q 

9 

$ 35,000 

0.6% 




Avg: 

10.1% 
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The standard contingency on a new construction project is typically five 
percent. These data show that the QBS process appears significantly better than 
the non-QBS for this small sample. The QBS projects on the average paid 5.6% 
less for changes. This seems logical. The better qualified the A/E firm, the less 
design errors should be expected during a project. Naturally, all the changes were 
not attributed to design errors, therefore in the future, the questionnaire should 
focus mainly on design changes. 

5.8 Owner’s Subjective Analysis 

The final analysis is the subjective project perception from the owner’s 
perspective. Since these are only from the owners perspective, there will be some 
bias. In future studies, perhaps a ranking of the owner and project can be done by 
the design firm as well. In table 5.8, column (3) are the negative and positive 
factors that influenced the project, such as external, owner controlled or project 
specific as discussed earlier in section 4.4. See appendix A, question 2.1.7 for a 
complete breakdown of these factors. Column (4) provides the design ranking on 
a scale of 1-5 with 1 being very unsuccessful and 5 being very successful. Column 
(5) is the overall ranking of the project from 0 to 100. 
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Table 5.8 Owner Subjective Analysis 


Project 

Q/C 

Factors (-,+) 



(1) 

( 2 ) 

_(3)_ 

(4) 

(5) 

P001 

C 

■m 

3 

82 

P002 

C 

_(0. 13)_ 

3 

80 

P005 

c 

■m 

5 

92 

P006 

c 

_&0)_ 

1 

90 


ESI 

K&saxnri 

3 

86 

P003 

Q 

■QKni 

3 

65 

P004 

Q 

(«. 3) 

4 

72 

P007 

Q 

(4, 6) 

4 

92 

P008 

Q 

_&*)_ 

4 

65 


Avg 

■m 

3.75 

74 


The QBS A/E firms ranked higher than the non-QBS in terms of design 
effectiveness, however, the projects from the non-QBS ranked higher than the 
QBS. The non-QBS projects did have more positive and less negative factors than 
the QBS. 


5.9 Evaluation of Data Collection Method 

The questionnaire method of collecting data takes a long time, especially 
since most professionals are using voice mail and personal contact is difficult. 
Initial contact can take three to four calls. Once the point of contact is located, a 
high degree of promoting the research project is a key to quick response. The 
response rate for this project was approximately: 25 percent said no; 25 percent 
returned within the specified time stated in the initial request, and 50 percent took 
multiple reminders. Therefore, it is very important in the initial contact to be 







































prepared. In short, the researcher should sound enthusiastic, search for common 
links, be concise, and get a firm commitment. 

The questionnaire on the average took approximately 45 minutes to 
complete by the respondent. Some of the problems that occurred during the 
collection of data can be improved by modifying the questionnaires. The author 
recommends changing the following questions: 


Project Performance Questionnaire 

a. Current: 

2.1.5 What is the approximate size or design capacity of this facility? 
(EXAMPLES:: gross square feet, number of beds, etc.) 


New: 

2.1.5 What is the approximate size of the facility?_SF 

Reason: 

Design capacity focuses on the industrial type project. 

b. Current: 

2.2.1 What was the capital cost breakdown, by the following major cost 
categories, for the estimated cost at the time of major funding authorization and 
the actual final cost of the project? In order to assist you in completing this 
section, guidelines for selected cost categories follows: 

Owner Costs : The direct owner incurred costs, excluding procured equipment or 
any subcontracts. 

Owner Procured^ Equipment/Materials : The costs associated with owner 
procurement of cmy equipment or materials inclusive of any capitalized 
subcontract costs (i.e., procurement by a subcontractor on an owner's purchase 
order). 



Total Project Cost 

Estimated Cost 
* 

Actual Cost 

Owner Costs 



Owner Procured Equipment/Material 



Engineering & Design Services 



Construction Contractor Equip, Material 
& Labor 



Commissioning, Turnover, & Startup 



Contingency 


xxxxxxxxx 

X 

Other 



Total Project Cost 




* at Authorization/Appropriation 


New: 

2.1.1 What is the cost breakdown of the project: 

Estimated Actual 


a. Construction _ 

b. Land/Capital _ 

c. Design _ 

Describe the design services that are being provided: 


d. Other 


e. Total Cost:_ 

Reason: 

Some of the factors in the original question are more for industrial type 
projects. Also, the description of the design is necessary to ensure the research is 
comparing the same amount of services. For example, if one design firm is only 
providing drawings while the other is providing additional site visits, etc., then it is 
not fair to compare the two. 













c. Current: 


2.4.1. What was the total approximate number of change orders issued (< including 
engineering and construction )? 


2.4.2. What was the total dollar amount of all change orders? 

$ _ 

( approximate, if necessary). 

2.4.3 What was the net change in the completion date resulting from change 
orders?_months 

Did the net impact of the changes increase or decrease the length of the original 
project duration? (Please check only one answer) 

[ ] Increase [ ] Decrease [ ] No change in project duration 


New: 

2.4.1 What was the total approximate number and cost of change orders issued: 

Number Cost 


Due to Design Errors 
Due to Unforeseen conditions 
Due to Owner 
Other 


Total 


2.4. 2 Omit 

2.4.3 What was the net change in the completion date resulting from change 
orders?_months. [ ] increase [ ] decrease 




Reason: 


Data is in a more useful format. Also, it can specifically isolate the design 
category which is the focus of the study. 

d. Current: 

2.5.1 What percent of design capacity (usable space, or capacity) was planned or 
anticipated at the time this project was authorized and actually and actually 
obtained 6 months after substantial completion? 

2.5.2 What percent of faciltiy utilization was planned or anticipated (at the time 
this project was authorized) and actually obtained at 6 months after the end of 
startup? 


Planned Obtained 

a. 6 months % _ % 


b. If planned utilization differed from that actually obtained, what were the main 
reasons for the differrence? (Please check all appropriate box(es)) 



Reason 

Reason 

[ ] 

Availability of facility 

[ ] Maintainability 

[ ] 

Quality 

[ ] Availability of users 

[ ] 

Performance 

[ ] Market Demand 

[ ] 

Other (please specify) 



2.5.3 Have there been any unanticipated renovations since substantial completion? 
(please check only one answer) 

[ ] No - (If no, please skip to the next question below) 

[ ] Yes - (Ifyes, please answer the following questions, 3a, 3b, 3c). 

a. What was the cost of the renovations? $_ 

b. How long did the renovations take? _months 



c. Why were these renovations needed? {Please check all that apply) 



Reason 


Reason 

[ ] 

Facility Function 
Modification 

[ ] 

Regulatory Change 

[ ] 

Operability 

[ ] 

Design Error 

[ ] 

Maintainability 

[ ] 

Increase Capacity 

[ ] 

Construction Error 

[ 1 

Other {Please specify) 


New: 

2.5. 1 - 2.5.3 Omit 
Reason: 

Questions in this section pertain to industrial type projects. 

e. Current: 

2.7.2 We would like to know your overall opinion of how well the design 
professionals performed on this project, taking into consideration all the areas that 
we have just covered. On a scale of 1 to 5, with 1 being very unsuccessful and 5 
being very successful, please provide us with your rating of the overall effort. 

1 2 3 4 5 

What are your main reasons for your assessment of the design professional? 

New: 

2.7.2 We would like to know your overall opinion of how well the design 
professionals performed on this project, taking into consideration all the areas that 
we have just covered. On a scale of 1 to 10, with 1 being very unsuccessful and 
10 being very successful, please provide us with your rating of the overall effort. 

123456789 10 
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What are your main reasons for your assessment of the design professional? 


Reason: 

Need to have more variance in the scale. Most ratings were 3 or 4. 

Service Acquisition Questionnaire 

a. Current: 

3. Check the one statement that best describes the timing of the cost proposal 
submission by the service provider to the owner: 

[ ] Submitted as a bid, with no pre-qualification. 

[ ] Submitted concurrently with firm’s qualifications. 

[ ] Submitted after pre-qualifying, but prior to selection. 

[ ] Submitted after pre-qualifying, and after selection.. 

[ ] Other (Please specify) 


New: 

3. Check the one statement that best describes the timing of the cost proposal 
submission by the service provider to the owner: 

[ ] Submitted as a bid, with no pre-qualification. 

[ ] Submitted concurrently with firm’s qualifications. 

[ ] Submitted after pre-qualifying, but prior to selection. 

[ ] Submitted after pre-qualifying, and after selection. 

but prior to contract award/negotiations. 

[ ] Other (Please specify) 



Reason: 


There needs to be a distinction between contract award and selection. This is the 
premise that this study is based on. Does the A/E firm get selected to negotiate 
before the owner sees any pricing data? If the answer is yes then it is non-QBS. If 
the answer is no then it is QBS. 



Chapter 6: Conclusions and Policy Implications 

6.1 Conclusions 

Why is it necessary to always pick the best design firm? If a firm is 
professionally qualified for a specific project, then is it qualified to do the job? The 
“Best Value” method seems to be widely used within the construction industry. 
This makes common sense and is the way most consumers think, whether it be 
choosing a service or buying a product. People will normally choose their services 
based on quality but within their budget. The incremental value of quality service 
must be equal to the incremental cost associated with paying more for that benefit. 
At the same time, a marginal performer just above the qualification line is not the 
optimum solution either. 

The Brooks Act was passed over 25 years ago with the philosophy of 
choosing the best first and then negotiating the price with a 6 percent statutory 
limitation. The design costs are broken into two parts which included the initial 
design fees to produce the drawing and specifications and then the engineering 
field services. The combination of these two can easily exceed 6 percent. 

It is time to take another look at the Brooks Act, and the evaluators must 
use statistical data to back up any decisions. There is a new era evolving in the 
design industry, as A/E firms from other countries are entering the US market and 
are able to compete quickly because they are producing the same quality of 
drawings for less money. That is probably one of the reasons why much of the 
private industry is beginning to see a shift towards “best value” engineering 
acquisition method. 




Realizing that this is only a pilot study of eight projects and a more in- 
depth investigation with more projects picked at random should be conducted, the 
following conclusions were apparent from this limited sample: 


• Half of the projects selected were non-QBS, and none were 
competitive low bid. However, several used price and qualification 
to select. 

• The percentage of design cost of non-QBS projects are 20 percent 
less than QBS projects. 

• There was no significant difference in cost per square foot between 
QBS and non-QBS, including design and construction cost. 

• The non-QBS projects completed 26 percent more on schedule 
than the QBS projects. 

• The QBS projects experienced less design changes than 
non-QBS. 

• There was no direct correlation with design ranking and overall 
project performance. 

• Both questionnaires need to be modified for future studies to focus 
more on design issues and to simplify the questions. Suggestions 
are given in this thesis. 

The null hypothesis for this study is that there is no significant relationship 
between project performance and the method of selection of the design service 
provider. Only a few factors were looked at in this study and the study is 
inconclusive although it is intriguing at the same time. This pilot study is a good 
indicator of a possible change that is occurring within the private sector. 




Specifically, this was noticeable by observing the results of a small sample of 
owners hiring A/E firms. However, there is no clear solution at this point and a 
solid study researching the benefits and shortfalls of A/E acquisition is warranted. 

6.2 Policy Implications 

Why continue with business as usual when the times are quickly changing? 
As mentioned earlier, there have been improvements to the FAR with the Federal 
Streamlining Act. Also Design/Build is another example of how the design 
selection method is transitioning. The Government spends millions of dollars on 
outsourcing A/E firms. Why not take a thorough look at the Brooks Act and 
address the applicability of moving into the 21st Century with the current system. 

The overwhelming policy implication is that there is no clear cut solution 
and there should not be one. One possibility is to provide a system where 
combination of qualifications of an A/E firm and complexity of the project should 
determine who is selected. As the complexity of the project increases so does the 
screening process for selecting an A/E firm. However, cost should always be a 
factor. This brings the concept back of choosing the “best value.” Not only does 
this enhance competition, but it will be the trend in the private construction 
industry into the future. Hopefully, the study that was piloted in this thesis will be 
performed in the future and provide guidance for change. 
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Appendix A: Project Performance Questionnaire 




# 


# 


PROJECT PERFORMANCE 
QUESTIONNAIRE 

Building Version 




The University of Texas at Austin 




15 Aug 96 


* 
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Project Performance Questionnaire 

Organization Name: ____ 

Point of Contact: 

1. Name:_ 

2. Title:_ 

3. Address: ____ 

4. Tel. No: _Fax No: _ 

General Project Information: 

1. Project Name:_______ 

(Is the above name for the project correct? If not, please correct it on the line below.) 


2. In what town or city is the project located? 


In what state?_ 

3. What type of facility is this project? (Please check only one) 


[ ] Jail/Prison 

[ ] Retail 

[ ] School 

[ ] State/County/City government 

[ ] Office Building 
[ ] Correctional Facility 
[ ] Maintenance Facility 

[ ] Other (please specify) 


4. What are the primary products or services produced by this facility?_ 

5. What is the approximate size or design capacity of this facility? _ 

(EXAMPLES:: gross square feet, number of beds, etc. ) 

6. Which of the following best describes the site on which this project was built? If more 
than 25% of the project was a renovation, please classify the project as a 
Retrofit/Expansion. (Please check only one answer) 

[ ] New construction [ ] Other (please describe) 

[ ] Retrofit/Expansion __ 


7. In the table below, circle the one word (negative, none, or positive) that best represents 
the overall influence that factor had on the project’s performance. After checking the one 
best word, check the box under all categories ( time, cost, quality, or other) affected by 
that factor. Factors left blank are assumed as not affecting the project. 

“Negative " signifies the factor negatively affected the project , causing time delays, increased cost, or reduced 
quality. “None ” signifies the factor did not affect the project, and had no influence on time, cost or quality. 

“Positive " signifies the factor positively affected the project, causing time acceleration, reduced costs, or 
increased quality. 






FACTOR 


Externally driven factors 


1. competitive const, market 


2. local conditions 


3. regulations or permits 


4. labor strikes 


5. material availabili 


6. abnormal weather 


7. unrest insurrection, war 


8. acts of god 


Owner controlled events 


re-proiect plannin 


10. contracting strate 


11. design reviews 


12. constructabili 


13. construction manager 


14. time constraints 


15. use of incentives 


16. financing or fundin 


17. partnerin 


Project specific factors 


18. remoteness of the site 


19. complexity of project 


20. new technologv/desi 


21. first of its kind 


23. 


24. hazardous materials 


rience 


26. proclivity to claim 


27. attitude 


28. safety practices 


Other (specify) _ 

29. 


Categories Affected 

(Check all that apply) 

Time 

1 Cost 

1 Oualitv 

Other 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

0 

D 

0 

□ 

□ 

□ 

□ 

□ 

Time 

1 Cost 

Quality iQther 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

Time 

1 Cost 

1 Quality 

Other 

□ 

□ 

a 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

n 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 


Time | Cost | Quality | Other. 


□ □ □ □ 


EXPLANATION: (Ifyou indicated that any of these factors affected the project, please 
indicate it by number and briefly explain below. Use the back of the page if necessary): 
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8. Please identify the prime design firm: 


Name of Firm:___ 

Address:_ 

Primary Contact: 

Title:_ 

Tel. No: Fax No: 

2.2. Cost Information : 

1. What was the capital cost breakdown, by the following major cost categories, for the 
estimated cost at the time of major funding authorization and the actual final cost of the 
project? In order to assist you in completing this section, guidelines for selected cost 
categories follows: 

Owner Costs : The direct owner incurred costs, excluding procured equipment or any 
subcontracts. 


Owner Procure d. Equipment/Materials : The costs associated with owner procurement 
of any equipment or materials inclusive of any capitalized subcontract costs (i.e., 
procurement^ by a subcontractor, on art owner's purchase order). _ 


Total Project Cost 

Estimated Cost * 

Actual Cost 

Owner Costs 



Owner Procured Equipment/Material 



Engineering & Design Services 



Construction Contractor Equip, Material & 
Labor 



Commissioning, Turnover, & Startup 



Contingency 


xxxxxxxx 

XX 

Other 



Total Pro ject Cost 




* at Authorization/Appropriation 


2. 


In terms of construction bidding, what were the : 


Number of bidders: 


Low bid: 


Second low bid: 


High bid: 



OR 

attach the construction bid list for the project 














2.3. Schedule Information: 


1. What was the date of major funding authorization?_ 

2. What was the planned duration of the execution schedule (from project authorization 

to substantial completion) at project authorization (in months)?_months. 

3. What was the actual date of substantial completion? _ 

4. If there were any schedule overruns greater than 1 percent of total project duration, 
please indicate the reason(s) in the appropriate box(es) below by supplying the duration(s) 
of the change(s) {Please check all that apply.) 


Delay Weeks. 

[ ] Scope/Design Change _ 

[ ] Labor Shortage _ 

[ ] Contract Dispute _ 

[ ] Weather _ 

[ ] Strike _ 

[ ] Material Shortage / Delivery _ 


Delay Weeks. 

[ ] Funding Change _ 

[ ] Regulatory Change _ 

[ ] Equipment Availability _ 

[ ] Construction Productivity _ 

[ ] Engineering Productivity _ 

[ ] Other {please specify) _ 


5. If you checked “Scope/Design Change” in 4., above, please describe the change(s) 


2.4 Change Information : 

1. What was the total approximate number of change orders issued {including 
engineering and 

construction )?_ 

2. What was the total dollar amount of all change orders? 

$ _ 

(i approximate, if necessary). 

3. What was the net change in the completion date resulting from change orders? 

_months 

Did the net impact of the changes increase or decrease the length of the original 
project duration? (Please check only one answer) 

[ ] Increase [ ] Decrease [ ] No change in project duration 

4. Were there any individual changes after project authorization that exceeded 1 percent 
of the project budget? (Please check only one answer) 

l ] No 

[ ] Yes - {If "Yes", what were the total cumulative effects and the direction of these changes 
on): 



a. Cost - $ _[ ] Increase or [ ] Decrease 

b. Schedule -_months [ ] Increase or [ ] Decrease 

c. How many changes were 1 percent of the original project budget or greater? 

d. What were the reasons for the changes? ( Please check all that apply) 


Reason Amount 

(%) 

User Change - 

Schedule Change - 

Weather - 

Differing Site - 

Conditions 

Estimating Error - 

Scope & Design - 

Changes 


Reason Amount(%) 

Funding Change _ 

Regulatory Change _ 

Strike _ 

Design Error 

Market Change 

Other (Please _ 

specify) 


Operating Information: 

1. What percent of design capacity (usable space, or capacity) was planned or anticipated 
at the time this project was authorized and actually obtained 6 months after 
substantial completion? 

Planned Obtained 
a - 6 months - 0// ° - 0// ° 


2. What percent of facility utilization was planned or anticipated (at the time this project 
was authorized) and actually obtained at 6 months after the end of startup? 

Planned Obtained 
a * 6 months -% -% 


b. If the planned utilization differed from that actually obtained, what were the main 


reasons 

; for the difference? ( Please check all appropriate box(es)) 


Reason 


Reason 

[ ] 

Availability of facility 

[ ] 

Maintainability 

[ ] 

Quality 

t ] 

Availability of users 

[ ] 

Performance 

[ i 

Market demand 

[ i 

Other (please specify) _ 




3. Have there been any unanticipated renovations since substantial completion? (please 
check only one answer) 

[ ] No - (If no, please skip to the next question below.) 

[ ] Yes - (If yes, please answer the following questions, 3a, 3b, 3c). 

a. What was the cost of the renovations? $_ 

b. How long did the renovations take?_months 

c. Why were these renovations needed? (Please check all that apply) 



Reason 


Reason 

[ ] 

Facility Function 
Modification 

[ i 

Regulatory Change 

[ ] 

Operability 

[ i 

Design Error 

[ ] 

Maintainability 

[ i 

Increase Capacity 

[ ] 

Construction Error 

[ ] 

Other (Please specify) 

[ ] 

Improve Efficiency 




2.6 Other Information 


1. Did the project have any claims that required arbitration, litigation, or mediation? 
[ ] No - (If no, please skip to the next question below.) 

[ ] Yes - (If yes, please answer the following question, la). 

a. If so, please indicate the magnitude and provide details. 


2. Were there any protests regarding selection of A/E's? 

[ ] No - (If no, please skip to the next question below.) 

[ ] Yes - (If yes, please answer the following question, 2a). 

a. If so, please elaborate? 





2.7 Subjective Evaluation of the Project 

1. The following questions are intended to subjectively evaluate the facility. (Read the statement about the project, 
then provide a response to the statement on a scale of 1 to 5, with 1 meaning you strongly disagree with the 
statement ; 3 meaning you neither agree or disagree and 5 meaning you strongly agree with the statement. DK 
stands for “Don 7 Know"). 


a. 


b. 


d. 

e. 

f. 
& 

h. 


j- 

k. 

l . 

m. 

n. 


In general, project participants worked well together. I 

Examples of project participants are owners, customers, design contractors, and 
consultants. The participants worked together toward the common goal of 
successfully designing the project. 

Emphasis was placed on identifying and satisfying the needs of the customer by the 1 
design professional(s). 

This project was characterized by high quality, professional performance by project 1 
participants. 

Innovative design solutions were used to solve project problems and provide a state- 1 
of-the-art facility 

Lessons learned from previous construction projects were incorporated into this 1 
project during the design phase. 

The facility design is aesthetically pleasing. 1 

The goals to maintain or improve the quality of the environment were met or 1 
exceeded by this project (Goals should include organization and/or regulatory) 

The completed facility has provided a safe workplace since it was placed into 1 
operations. 

The goals concerning ease of facility operation were met or exceeded. (Examples of 1 
ease of operation goals are operating staff size and overtime.) 

This project met or exceeded its goals concerning the number of days it was available 1 
for operation in a year. 

There has been little need for any major, unplanned facility improvements or changes 1 
since completion of this project 

The goals concerning the ease of maintenance were achieved by the execution of this I 

project 

The experience of the design professional was adequate to perform the design of this I 

project 

The design professional fostered an effective level of communications during the 1 
course of the design 
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2. We would like to know your overall opinion of how well the design professionals 
performed on this project, taking into consideration all the areas that we have just 
covered. On a scale of 1 to 5, with 1 being very unsuccessful and 5 being very 
successful, please provide us with your rating of the overall effort. 


1 2 3 4 5 


What are your main reasons for your assessment of the design professional? 





3. In ranking this project’s success against all projects you are familiar with, in what 
percentile would you place this project? (Circle, or "x” the appropriate percentile on the 
scale below.) 

Worst Best 


0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 
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Appendix B: Service Acquisition Questionnaire 



SERVICE ACQUISITION 
QUESTIONNAIRE 


The University of Texas at Austin 




Project Questionnaire 


1.1. Project Name: 

Prime Design Firm:_ 

Address:____ 

City:_State_Zip_ 

Primary Contact: _ 

Title:____ 

Tel. No: _Fax No:_ 

2.1 Service Provider Selection Factors 

1. Check the one statement that best describes the service provider’s fee for this project: 

[ ] Fixed price. 

[ ] Reimbursable costs with fixed profit. 

[ ] Reimbursable costs with guaranteed maximum price. 

[ ] Reimbursable costs plus incentive payment. 

[ ] Multiple of Hourly Rate(s). 

[ ] Percent of Construction. 

[ ] Cost plus % of Cost 
[ ] Other (Please 

specify__ 

2. Check the one statement that best describes the degree of cost negotiation pressure 
by the owner on the service provider: 

[ ] None [ ] Some [ ] Intense [ ] Inflexible 

3. Check the one statement that best describes the timing of the cost proposal 
submission by the service provider to the owner: 

[ ] Submitted as a bid, with no pre-qualification. 

[ ].Submitted concurrently with firm’s qualifications. 

[ ] Submitted after pre-qualifying, but prior to selection. 

[ ] Submitted after pre-qualifying, and after selection.. 

[ ] Other (Please specify) 
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4. Check the one statement that best describes the format of the cost proposal: 
[ ] Lump sum. 

[ ] Hourly Rates plus multiplier. 

[ ] Hours (not dollars) provided. 

[ ] Other (Please specify) 



5. Check the one statement that best describes the degree of competition among the 
service provider’s seeking to be selected for this project: 


[ ] Sole source 

Pre—aualified? 

[ ] Yes [ ] No 

[ ] 2 or 3 competitors 

[ ] Yes 

[ ] No 

[ ] 4 or 5 competitors. 

[ ] Yes 

[ ] No 

[ ] 6 + competitors 

[ ] Other (Please specify) 

[ ] Yes 

[ ] No 


6. Check the one statement that best describes the nature of the pre-qualification, if 
any, that was required, to be considered for this project 

[ ] No pre-qualification required. 

[ ] by owner’s invitation only. 

[ ] open to any interested firm. 

[ ] Other (Please specify) 


7. Check each statement below that describes the qualification(s) considered in 
selecting a service provider for this project, and indicate the degree of importance that it 
played in the selection: 


Importance: Not 

[ ] Technical expertise in this type project. 1 
[ ] Technical expertise in similar type 
projects. 1 

[ ] Experience in this type of project 1 
[ ] Experience in similar types of projects 1 
[ ] Immediate availability to begin this 
project. 1 

[ ] Other (specify)_ I 


somewhat 

2 3 4 

2 3 4 

2 3 4 

2 3 4 

2 3 4 

2 3 4 


Very 

5 

5 

5 

5 

5 

5 


8. On what primary basis was the service provider chosen? 

[ ] Qualifications [ ] Cost [ ] Don’t Know [ ] Other 

(specify)_ 
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